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Abstract 

We analyze the behavior of closed product-form queueing networks when the number of customers grows 
to infinity and remains proportionate on each route (or class). First, we focus on the stationary behavior 
and prove the conjecture that the stationary distribution at non-bottleneck queues converges weakly to the 
stationary distribution of an ergodic, open product-form queueing network. This open network is obtained 
by replacing bottleneck queues with per-route Poissonian sources whose rates are determined by the solution 
of a strictly concave optimization problem. Then, we focus on the transient behavior of the network and use 
fluid limits to prove that the amount of fluid, or customers, on each route eventually concentrates on the 
bottleneck queues only, and that the long-term proportions of fluid in each route and in each queue solve the 
dual of the concave optimization problem that determines the throughputs of the previous open network. 

1 Introduction 

Complex systems such as communication and computer networks are composed of a number of interacting 
particles (or customers) that exhibit important congestion phenomena as their level of interaction grows. 
The dynamics of such systems are affected by the randomness of their underlying events, e.g., arrivals of 
units of work, and can be described stochastically in terms of queueing network models. Provided that these 
are tractable, they allow one to make predictions on the performance achievable by the system, to optimize 
the network configuration and to perform capacity-planning studies. These objectives are usually difficult 
to achieve without a mathematical model because real systems are huge in size; e.g., Urgaonkar et al. [34j . 

The focus of this paper is on the well-known class of closed queueing network models introduced in Kelly 
[18| and Baskett et al. [3]. Specifically, a fixed number of customers circulate in a network following given 
routes. A route is a sequence of queues (or stations) that forms a cycle in the network. In terms of the amount 
of service required on each queue, users belonging to the same route are statistically equivalent. In contrast, 
users belonging to different routes can be statistically different. The stationary probability distribution of 
the numbers of customers in each queue of this class of queueing networks has the product-form property, 
which formally means that it can be written as the product of simple terms associated to each queue up to 
a normalizing constant. This surprising property represents a big step forward for the understanding of the 
stationary behavior of these queueing networks. However, due to the quick growth of the state space and 
despite the attention devoted to this problem during the last decades, the computation of the normalizing 
constant remains a notoriously-difficult task especially when the number of customers is large. This limits 
the application of these models to optimization and dimensioning studies of real systems. 

During the last decades, several approaches have been investigated to assess the stationary behavior of 
closed product-form queueing networks with a large number of customers. A large body of the literature 
aims at developing exact algorithms for the efficient computation of the normalizing constant or stationary 
performance indices such as throughputs and queue lenghts; see, e.g., Reiser and Kobayashi [26] . Reiser and 
Lavenberg [27], Harrison and Coury [14] . Casale [6] and the references therein. To the best of our knowledge, 
no exact algorithm has a running time that is polynomial with respect to the numbers of routes, customers 
and queues. Motivated by this difficulty, a number of alternative analyses emerged in the literature for the 
stationary behavior. These mainly consist in: 

• Using the mean value analysis (MVA) by Reiser and Lavenberg [27] to develop iterative or fixed-point 
algorithms; e.g., Schweitzer [29], Chandy and Neuse [8], Pattipati et al. [24], Wang et al. [36]. While 
these techniques improve the running time of MVA, there is no guarantee that they converge to the 
exact solution except for particular cases. 

• Developing efficient bottleneck identification techniques; e.g., Schweitzer [3D], Schweitzer et al. |31j . 
Casale and Serazzi [7], Anselmi and Cremonesi pQ. This approach aims at reducing the network size 
by ignoring the impact of the stations that have a minor influence on the overall performance. 
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• Studying the network behavior in some limiting regime, e.g. for the instance of Jackson networks see 
Goodman and Massey [12]. Our closed queueing context, one can assume the existence of a M/M/oo 
queue within the network and allow the total number of jobs, say n, grow in proportion to its service 
times; McKenna and Mitra [55], McKenna |22| . Berger et al. [3]. Another possibility is to let n grow 
to infinity in proportion to the number of stations; Knessl and Tier [191 120] , Finally, one can also 
study the network behavior when n grows to infinity keeping fixed the proportions of customers in 
each route; Pittel [25], Schweitzer [30] , Balbo and Serazzi [2], Walton [33], Walton et al. [16] . Anselmi 
and Cremonesi ,1 , George et al. [11] . 

The focus of this paper is on the last approach where n — > oo keeping fixed the proportions of customers 
in each route. In this limiting regime, it is known that some queues, called bottlenecks in the following, 
increase their backlog proportionally to n, see Pittel [25] and Walton et al. [16] . and uniquely determine 
the throughput of customers along each route by means of a concave optimization problem; Schweitzer [3D], 
Walton [35]. Interestingly, this optimization problem coincides with the utility optimization problem that 
determines the fractions of bandwidth (or rates) allocated to multiple classes of concurrent Internet flows 
(or end-to-end Internet connections); see Kelly et al. [IT] . Srikant [33) . On the other hand, the amount 
of backlog in each non-bottleneck is strictly bounded by 0(n) but in general its limiting behavior is not 
known. There is numerical evidence to support the conjecture that the limiting stationary distribution of 
each non-bottleneck queue is geometric; Balbo and Serazzi [2J. Such convergence was proved by Pittel [25] 
under the assumption that the mode of the stationary distribution is unique; see also Gordon and Newell 
[13] . Lipsky et al. [21], Anselmi and Cremonesi pQ. This assumption, for instance, is not satisfied when the 
number of bottlenecks is less than the number of routes but greater than one, which is a natural scenario in 
real networks. As we shall explain, one of the principle contributions of this paper is to prove this conjecture 
by removing the unimodal assumption. 

The above conjecture provides an accurate and efficient approximation to the per-route mean delay (or 
cycle time) of a network with n customers that takes into account the contribution of non-bottlenecks (a 
number of techniques only consider the mean delays at the bottlenecks; see, e.g., Schweitzer et al. [31] 
for an overview). The impact on mean delay of non-bottlenecks becomes non-negligible if their number 
prevails significantly, and this is often the case; see Casale and Serazzi [7], Anselmi and Cremonesi [T]. This 
approximation allows for the direct development of efficient optimization frameworks able to address, for 
instance, data-center consolidation problems, where the objective is to reduce the cost and the size of a 
data-center while guaranteeing a given performance level. Furthermore, it provides a good initial guess for 
the iterative or fixed-point algorithms mentioned above. 

1.1 Our contribution. 

Following the approach considered by a large body of the literature, we are interested in the behavior of closed 
product-form queueing networks when the number of customers in each route grows to infinity proportionally. 
This is mainly motivated because real networks are populated by a large number of customers; e.g., Urgaonkar 
et al. 134] . Our objective is two-fold and consists in analyzing the problem from two contrasting standpoints. 

The first part of this paper focuses on the stationary behavior of these networks. We prove the conjecture 
that the stationary distribution of non-bottlenecks converges weakly to the stationary distribution of an 
ergodic, open product-form queueing network. This open network is obtained by replacing bottlenecks with 
per-route Poissonian sources whose rates are determined by the solution of a strictly concave optimization 
problem. 

In contrast to the first part, the second part of this paper focuses on the transient behavior. Starting 
from any arbitrary distribution of customers on queues and using justified fluid-limit equations, in our second 
contribution we prove that the amount of fluid, or customers, on each route eventually concentrates, as time 
increases, on the bottleneck queues only and that the (long-term) proportions of fluid in each route and in 
each bottleneck solve the dual of the concave optimization problem that determines the throughputs of the 
open network described above. Our proof for closed queueing networks uses an entropy Lyapunov function 
similar to the one used by Bramson [5] to establish convergence properties of the fluid-limit equations of 
open queueing networks. 

The technical difference behind the two contributions above is the order in which the limit in the number 
of customers and the limit in time are taken. In the second part, the limit in the number of customers is 
taken before the limit in time, and vice versa. In stochastic systems, these two limits do not commute in 
general, but for the class of queueing networks investigated in this paper we prove that they do. Taking the 
limit in the number of customers first provides a natural way to look at the evolution of a network populated 
by a large number of customers and, by subsequently taking a limit in time, we justify fluid model arguments 
within the queueing literature. The second result proven in this paper, thus, increases the robustness of 
the approach taken in the first part, which has been followed by several researchers as reference above, 
and furthermore, it can be also seen as a queueing theoretic analysis of the utility optimization found in 
congestion control protocols; e.g., Kelly et al. [17] and Srikant [33] . 
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1.2 Organization. 

In Section [5] we introduce the models considered in this paper. In particular, these include two Markov 
descriptions of a closed queueing network, relevant quantities such as bottleneck and non-bottleneck queues 
are defined, and an expression for a fluid model of a closed queueing network is given. In Section[3] we present 
the three main results of this paper: Section ETTl shows the asymptotic independence of non-bottleneck queues 
in the large-population limit; Section [3.21 shows the convergence of the Markov closed queueing network to 
a fluid solution; and Section T3. 31 states that a fluid solution converges to the set of bottlenecks in a way that 
minimizes a certain entropy Lyapunov function. In Section 0] we prove the main results stated in Section [3] 
We respectively prove the results of Sections 13. II 13.21 and 13.31 in Sections |4. II I4.2l and l4.3l 



2 Closed queueing networks models. 

We consider closed, multi-class queueing networks in the sense of Kelly [18] and Baskett et al. [3]. The 

set J C N denotes the set of queues (or stations) and we let J — \J\. The set X C 2 J denotes the set 

of routes (or classes) and we let I = \X\. A route is a sequence of queues visited by a customer during 

one cycle of the network. We assume, for simplicity, that each customer visits each queue at most once 

within a cycle of the network. Within each route i — we associate a route order 

For k = 1, ki — 1, a customer departing queue f k will next join queue f k+1 and a customer departing 

queue jl. will join queue queue j\. Unless otherwise specified, i will be used to index routes and j will be 

used to index queues. We assume that a constant number of customers circulate along each route of the 

network. We denote by n = {rii : i G X) G N 7 the population vector, i.e., the total number of customers on 

each route. When joining queue j, we assume that route- i customers require amounts of service that are 

independent and exponentially distributed with mean flj^- At each queue, we assume that customers are 

served at rate 1 according to a processor sharing discipline and customers joining a queue take a position 

uniformly at random in the queue. Thus, if a route-i customer does not join a queue j, i.e. j £ i, we may 

assume uj; =0. 
' j l 



2.1 Two Markov models of closed queueing networks. 

Firstly, we could describe the exact location of each customer in the queue according to its route type. Here 
the explicit state of a queue j would be a vector Sj — (ij(k) : k = 1, ...,rrij), where rrij is the number of 
customers in the queue and ij (k) is the route type of the customer in the k-th position. The explicit state 
of the network would then be the vector of each queue's state, s = (sj : j 6 J). We then let X(n) be the 
set of of the explicit states where the number of customers of each route type i is n; . 

Secondly, we could ignore positional information about customers within a queue, and instead, just 
consider the number of each route type at a queue. Here we let m = (rriji : j G J , i G X, j G i) be a network 
state, where rriji represents the number of route i customers in queue j. Thus, S(n) — {m : rriji =n%,i G 
X} is the state space of this Markov chain. It may be verified in a straightforward manner that this state 
space is finite and irreducible. 

Under the above assumptions, it is known (Kelly [18] and Baskett et al. [3]) that the stationary dis- 
tribution of being in state c, which we denote by 7r(s|n), or being in state m, which we denote Tr(m\n), is 
respectively 

rrij 

-inn - i n n ^ m,< , « e ( 2 .i) 

V ' j£J k=l y ' j£J 



where B(n) is the normalizing constant 



*<»)= E II (L .,) II"" 

m£S(n)j6J \ \ n ' J J i:j€i 



(2.3) 



To count the possible orderings of customers inside a queue, we use the multinomial coefficient 



rriji 



In the following, we mainly consider the stationary distribution (12. 2p . while the expression H2.1|) will be 
used in proofs. We remark that there are a large number of generalizations of our queueing network where 
(|2.2|l still gives the stationary number of customers of each route at each queue. Thus, our results generalize 
to such cases. For instance, we could generalize our processor-sharing discipline to the class of symmetric 
queueing disciplines with unit service capacity, see Kelly |18l Section 3.3]. We could also generalize service 
requirements to be independent with mean fijl and having a rational Laplace transform. If we keep the 
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assumption that service requirements are exponential and we assume that the state space of our Markov 
description is irreducible, then we could generalize to allow any service discipline that allocates service 
amongst customers in a way that does not discriminate between the route types of customers at the queue. 
Such service disciplines are described by Kelly |18l Section 3.1]. Finally, in all cases, we may elaborate the 
routing structure of our network; we may allow a route i customer to be routed through the network as a 
Markov chain (with finite expected exit time) whose states are determined by the set of queues so far visited. 
Although generalizations leading to stationary distribution (|2.2p are abundant, for simplicity of exposition, 
we do not explore these in further detail. 

2.2 Throughputs and bottlenecks. 

A key quantity of interest is the rate at which customers complete service on each route with respect to a 
reference queue (say j\ for route i). Let 

M«) = £ ^j^^mln) (2.4) 

meS(n): H 

m -i >0 

n 

be the throughput of customers on route i observed at queue j\ . In addition, let 

A.H^^^A^n) (2.5) 

be the load (or utilization) at queue j, i.e., the mean amount of work arriving at queue j per unit of time. 
Remark 1. The expression (|2.4|l gives the service completion rate of route i customers at queue j\, i.e., 
fj,ji it times the service rate devoted to these customers at that queue, i.e., m^ i /m^i , times the stationary 
probability that there are m customers in the network, i.e., 7r(m|n). Thus, this expression gives the mean 
rate (or throughput) for which route i customers leave queue j\. Note the route i throughputs at each queue 
j £ i must be equal, without loss of generality we consider j\ . 

A more concise expression for the per-route throughput, which is verified in AppendixfXl is given in the 
following lemma. 

Lemma 1. 

. . . B(n — ei) „_ 
A,(«) = -^, 

where B(n) is the normalizing constant l|2.3fl and e, is the i th unit vector in H+. 

In agreement with other works (e.g., Anselmi and Cremonesi [T], Balbo and Serazzi [2]), we define a 
bottleneck queue as a queue whose service is saturated. 
Definition 1. Queue j £ J is called bottleneck if and only if 

lim Aj(cn) = 1. (2.6) 

c — yoo 

We define the set J C J to be the set of bottlenecks and let J = \ J\. Similarly, we define J° — J\J to be 
the set of non-bottleneck queues and let J° = \ J'°\. 

In other words, a bottleneck is a queue whose load approaches the queue's unit service rate asc^ do. 
We will think of J° as the open part of the network and of J as the closed part of the network. 

2.3 Non-bottleneck queues and open queueing networks. 

We are interested in the probability distribution of non-bottleneck queues. For this reason, we consider 
queue-size vectors m° = (m°; : j G J° ,i G I, j € i) G 7L\ xI and 

7r°(m»= f ir((m,m°)\n), j£j°,i£l, (2.7) 

(rh,m° ) £tS(n) 

which defines the stationary probability that non-bottleneck queues are in state m° . 
We also define 

7Ta("1°) = Yi 7, J>A( TO °)> ( 2 - 8a ) 
36J° 

ra° G Zl° xl , A G K+, where 



The distribution tv, given by (|2.2[) . refers to the stationary distribution of a closed queueing network. In 
contrast, the distribution n% can be shown to be the stationary distribution of an open queueing network 
constructed on queues J° . Here customers arrive on each route as a Poisson process of rate A = (A» : i G I) 
and depart the network after receiving service at each queue on their route i n J° , see Kelly |18| and Baskett 
et al. 0. 
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2.4 Fluid Model. 



In order to study the transient behavior of our closed queueing network, we will analyze the following fluid 
model. 

Definition 2 (Closed queueing network fluid model). The processes m(t) = (rn,ji(t) : j £ J, i £ X, j £ i) 
and A(t) = (Aji (t) : j £ J , i £ I, j £ i) form a fluid solution ( or fluid limit) of our closed queueing network 
if they satisfy the following conditions: 

m J i i (t) = A Ji At) -A? At), (2.9a) 

J fc J fc — 1 J fe 

— (AjAt)-AjAs))<t-s, (2.9b) 

Aji(t) is increasing, (2.9c) 

ifmjtt) > then ^tlM = ( 2 .9d) 
dt uij 

y^mjjft) = m. (2.9e) 

Here, j £ J ,i £ J ,j £ i, t > s > 0. j'J «s i/ie fc" 1 queue on route i and is the queue before the 

k queue (we use the convention that the queue before j\ is jl )- 

The conditions (|2.9|l . thus, are the defining properties of a fluid solution and we observe that they 
are analogous to the ones used by Bramson [5l Formulas (2.3)-(2.6)], which hold for the open versions 
of the considered closed queueing network^]. In particular, conditions (|2.9a|l . (|2.9c|) . (|2.9efl are basic and 
relate queue lenghts in an obvious manner, condition (|2.9d|l is the property that defines a processor-sharing 
discipline, and condition (|2,9b[) takes into account the maximal processing rate of the system. 

We note that the condition (|2.9b|l implies that Aji is Lipschitz continuous. By (j2.9a.fl . this is also 
true for liiji. Lipschitz continuity implies absolute continuity, and therefore the processes Aji and mji are 
differentiable almost everywhere with respect to the Lebesgue measure. Throughout this document, the term 
for almost every will refer a set of real numbers whose complement has Lebesgue measure zero. Shortly, we 
will prove that the limit of the closed queueing network described previously satisfies the fluid model H2.9|) . 



3 Results on closed queueing networks. 

In this section, we present the three main results of this article. These are: Theorem [T] which states the 
independence of non-bottlenecks under a large-population limit; Theorem[2l which states that the stochastic 
process limit of a closed queueing network is a solution to the fluid equations (|2.9fl ; Theorem [3] which shows 
the convergence of the fluid solutions (|2.9[) to the set of bottleneck queues via a Lyapunov function argument. 



3.1 Independence of non-bottleneck queues. 

We now consider the limiting behavior of our queueing network with cn customers when c — s> oo. Given the 
constraints on the number of customers on each route, the distribution of customers at queues are certainly 
dependent variables; however, in classical open queueing systems, where all customers arrive and depart the 
network, the stationary distribution of queues can be shown to be independent; see Kelly [TS] and Baskett 
et al. [3]. We wish to demonstrate that non-bottleneck queues become independent as our closed queueing 
network congests. This fact has been conjectured and used in Balbo and Serazzi [2], and in this paper we 
present a proof. 

In our closed queueing network context and given (|2.7|) and (|2.8[) . in informal terms we wish to prove 
that for each n £ R+, m £ xl 

7r (m |cn) > 7Ta(w ), (3-1) 

c — ►oo 

for some throughput vector A £ M.+ which we must identify and for J° the set of non-bottleneck queues 
which must also be identified. Since the vector cn needs not to belong to , we consider cn + k c where 
{fcc}c£N is any uniformly bounded sequence such that cn + k c £ Z+. Our theorem can then be written as 
follows. 

Theorem 1. For n £ R{,m £ Z^° x/ 

n°(m°\cn + k c ) > ^A*(n)( m °) (3-2) 



1 Actually, the open queueing networks defined in Bramson [5], called head-of-the-line processor sharing networks, appear different 
from ours. In fact, upon completion of service at one queue, a class-ii customer becomes of class 12 with probability Pi 1 i 2 ■ However, 
one can easily build a mapping from one representation to the other. 
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where (A*(n), . . . , A|(n)) is the unique minimizer of the strictly-concave optimization problem 



(3.3a) 

j e J (3.3b) 

i £ I, (3.3c) 
and where J° , the set of non-bottlenecks, is given by the set of queues j such that 

y, A U n ) < 1 

The solution of optimization problem (|3.3[) . A*(n), is known in the literature as the proportionally-fair 
allocation (see Kelly [15]) and, interestingly, emerged independently as a model for the sharing of bandwidth 
among Internet connections (e.g., Srikant [33] ). For a detailed treatment of the relationship between closed 
queueing networks and the proportionally-fair allocation, see Walton [35| . 



maximize ni log Ai 

igZ 

subject to Ai/nji < 1, 

over Ai > 0, 



3.2 Existence of fluid limits for closed queueing networks. 

In Section 13.11 we analyzed the stationary probability distribution (|2.2|) of the closed queueing networks 
under investigation in the large-population limit. Now, we focus on the transient probability distribution 
in the large-population limit and then study the evolution in time of the system. In other words, the limit 
in time is now taken before the limit in the number of customers. In stochastic systems it is known that 
both limits are not interchangeable in general. The fluid limit, see Definition^ is a natural framework that 
allows for the analysis of such scenario. 

We consider a sequence of the closed queueing networks described in Section [2] In this sequence, the 
only variables that change are the number of customers on each route (the number of queues, routes, service 
distributions are kept fixed). We let the vector n £ be the proportion of customers on each route of the 
network. In the c th network of this sequence of closed queueing networks, there are cn + k c customers on 
each route, where {fc c } cg N is a bounded sequence of variables in such that cn + k c G 

We let Mji(t) be the number of route- i customers in queue j at time t of the c th closed queueing network. 
We let A5j (t) give the total number of route-i customers served by queue j by time t in the c th closed queueing 
network. From this, we define the rescaled processes 

B - JC M dot M^(ct) - def Afjjct) 

We wish to show that the vector processes M c and A c converge to a fluid solution. 

Theorem 2. The sequence of stochastic processes {(M c , A c } cS n converges in probability on uniforms on 
compact time intervals to a process (m, A) that satisfies the fluid solution equations (|2.9p . 



3.3 Convergence of the fluid solution. 

Having established the existence of a fluid solution, say (m, A), of our closed queueing networks, now our 
goal is to study its evolution in the long-term, i.e., rhtj = f dmy (t)/dt when t — > oo. In our main result, 
we show that the amount of fluid m eventually concentrates on the bottleneck queues only and that the 
long-term proportions of fluid in each route and in each queue solve the dual of optimization problem (13.31 1 . 

From the stationary distribution 7r(m|n), given by (|2.2|l . and under the same premise of Theorem[T] we 
can show that 

lim — log7r(cm|cn) = — /3(m) 

c— too C 

where 

|3(m) = ^^m ii log» (3.4) 
z — ' z — ' rrij 

Here, we have used the notation pa = Si. 

Remark 2 (Relative Entropy). A key quantity that will be useful in our proofs is the unnormalized relative 
entrop^ 

£>(p|k) d = 5> lo JS-. P,q€Mi. 
I 1 



2 This entropy is unnormalized because we do not enforce the condition that its arguments are probability distributions. 
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We notice that the function fit, given by ()3.5|) . is a linear combination of these unnormalized relative entropies 
for each queue. Furthermore, we note that if^^^Pi = Y2iQi> then we can renormalize and treat p and q 
as probability distributions. One can show that D(p\\q) > 0, using Jensen's inequality, and that D(p\\q) is 
minimized if p — q. 

Remark 3. It is an interesting observation that the rate of decrease of /3(m), the entropy of states, will be 
determined by the relative entropy between the rates of service; see Proposition below. Although distinct, 
this argument is similar to those argued for Markov processes by Spitzer ' L 32l and more recently by Dupuis 
and Fischer UOf . A further important reference is Bramson which shows, for the open versions of the 
closed queuemg networks considered in this paper, that the amount of fluid in non-bottlenecks converges to 
zero. 

In Bramson J5j[, the main argument behind the fluid convergence to zero for non-bottleneck stations 
relies on closing the open network with an additional (artificial) queue whose service rate equals the overall 
throughput of the network. All customer routes enter this additional queue and it is assumed that this queue 
never empties. This plays a significant role in regulating traffic to match that of the open network. So 
although a closed queueing network appears in that analysis, this additional queue introduces an important 
loss of generality. 

The function (3(m) forms a natural candidate for a Lyapunov function. We need to show that, for n G IR+ 
fixed, P(m(t)) decreases to its minimal value 

P* = minimize /3(m) subject to ^~]mji = n-i, i El. (3-6) 

We will consider the set of points attaining this minimal value 

M "= argmin /9(m) subject to '^_ l m i i ~ n '' i ^1- (3-7) 

The following theorem shows how the amount of fluid eventually distributes among network stations. 
Theorem 3. Form(t),A(t) a fluid solution (|2.9[) . 

(3(m(t)) \ /?*, as t — y co 

and, moreover, 

min \m(t) — m* \ — > 0, as t — > oo. 
In the above theorem, for the norm \m\ is the euclidean norm in bR XxJ . 



4 Proofs of main results. 

We now focus on proving the main results of this paper, namely, Theorem [TJ Theorem [5] and Theorem 
These are respectively proven in Sections 14. 11 14.21 and 14.31 

4.1 Analysis of non-bottleneck queues. 

We develop a proof of Theorem [1] Recalling the informal statement (|3.1[) . we need to verify that for each 
n G K+, m G Z^° x/ 

n°(m°\cn) s> TT/^(m°), (4-1) 

c— >oo 

for some A G M+ which we must identify and for J° the set of non-bottleneck queues which we must identify. 
Before verifying such a statement, we must identify the relevant throughput vector A and non-bottleneck 
queues J° . Fortunately, the following result, which was proven by Walton [35], characterizes A and J° . 

Proposition 1 ([35]). 

A*(n) = lim Ai(cn + fc c ), i 6 1 

c— »oo 

where A*(n) is the unique minimizer of the concave optimization problem (|3.3[1 and so that A;(cn + k c ) is 
defined on a point in its domain, {fc c } c gN is any bounded sequence in R + such that cn + k c G Z + . 
Consequently, j G J° iff 

A '( n ) < i 

Now, we show that (|3.2p holds. Before proceeding with this proof, we introduce a specific closed queueing 
network, which will helps us in the proof. Recall the closed queueing network defined on set J introduced 
in Section [2] Consider a queueing network defined exactly as in Section [2] except that queues J° are 
removed. The resulting queueing network has states m — (fhji : j € J,i G X,j £ i) G Z+ x/ ; if there are 
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n g 1+ customers on each route, this network has state space S(n) = {m : 5^ . rriji = tii,i £ I}, stationary 



distribution 



where 



w») - n ((^ e J n ^ ) , * * m, c«> 



<'>■>•■■ e n(L*; g< )n^) 



mgS(n) jgj" 

and stationary throughput 

A(n) = 

B(n) 

Proposition [T] holds for this network and, here, we would consider A*(n), the solution to the optimization 
maximize ^^riilogAi subject to hi/ flu < 1, 3 ^ J over Aj > 0, iel. 

In this optimization, all constraints that are not relevant to our solution A*(n) are removed. Thus, it is not 
surprising that the following lemma holds. 

Lemma 2. A*(n) = A*(n). 

We prove Lemma[2]in Appendix^] A direct consequence of Lemma[2]and Proposition [T] is the following. 

Lemma 3. 

Mn) = g( "~ e<) ► A*(n), i e X. 

We can now proceed to demonstrate that (|3.2p holds. Let us consider the equilibrium distribution 
TT°(m°\n). We have 



7r°(m°|n) = 7r((ra, m°)|n) 



(rh,m° ) £*S(n) 



(m,m°)£<S(n) 



n L."!« n-;/* - e n (L ':•.,.) n 



Pit 



B(n) B(n-n°) 

B(n) Bin) 11 \ \ m°, : j £ i. 



Here, n° = (n° : i £ X) is the number of route-i customers in non-bottleneck queues, i.e. n° = X^jginj" m ? i ' 
The third equality above follows by observing that the summation is over all the states where the number 
of non-bottleneck customers is n — n° . 

We now consider how the terms (a), (b) and (c), converge as we keep m° fixed and let n increase. 
Term (a) is easily dealt with as it does not depend on n. Term (b) will be shown to converge in the next 
proposition. Subsequently, term (c) will take a more in depth analysis. 

Term (a) represents the correct expression for the unnormalized stationary distribution of our open 
queueing network (see (|2.8[) ). except that we do not include the multiplicative term 

n n A *w ro|i =n A *« ? - 

3&J° i€Z 

As the following proposition shows, this is the limit of the term (b). 

Proposition 2. For n° £ n £ and {k c } ce m some bounded sequence such that cn + k c £ 7L\ 



B(cn + k c -n°) 



B(cn + fee) c->oo 



Proof. From Proposition [T] we have that 

B(cn + k' c — a 



B(cn + k' c ) c-+oo 



>A*(n), (4.4) 
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for any n £ R+ and any bounded sequence {k' c } ce fq such that cn+k' c £ Z+. Let K = £\ n°, Let e^m, e^^Q 
be a finite sequence of unit vectors and n(0), n(K) be a sequence of vectors in such that 



n(0) = 0, n(K) = n 

n(k) = n(k - 1) + e i(fe) , fe = l,...,Jf. 



Applying (|4.4p . we have that 
B(cn + k c -n°) _ 



n 



B(cri + fc e - ra(fc) - e t(fc+ i)) 
B(cn + k c - n(fc)) 



— ► n A »(fe)( n ) = n a » ( r 



as required. 



□ 



As the above proposition holds for any closed queueing network and because Lemma [3] holds, we can say 



that 



B(cn + k c — n°) -i-r . * , xn° rr rr . * , ■, 

\, ± i ' — ^ n a * n = n n a * n 

B(cn + k c ) c^oc 11 11 11 



(4.5) 



We now study term (c) in expression (|4.3[1 . We can note that this term is exactly the probability that 
all non-bottleneck queues are empty. 

Lemma 4. 

B{n) 



n({ mj = 0,Vj £ J°}\n) = 



B(n)- 



Proof. We have the following equations: 



n({m j =0,Vj€j°}\n)= £ * ]} f 



^ B(n) 11 

(0,m)6S(n) V 7 jgj 



^ B(n) n 

m£S(n) jej 



mjj : j £ % 



rriji : j £ i 



ii 



n/ 



mjj : j £ i 



11 ^' B(nY 



□ 



Although it is difficult to directly deal with events of the form \rrij = 0, Vj £ we can deal with 

events of the form {m,j > 0, Vj £ J'} where J' C J° . 

Lemma 5. For J' C J° , 



n({mj > 0,Vj G J"'}|cn + fc c ) > A H r 



where we define 



A » = E 



i:j£i 



A*(n) 



Proof. To prove this lemma, we consider the explicit stationary distribution of a closed multi-class queueing 
network l|2.ip . 

Recall s — (sj : j £ J) where Sj = (ij(k) : k — l,...,mj) £ T m i keeps track of the exact position 
and route type of each customer within a queue. Within this state representation, we can calculate the 
probability that the customer at the head of each queue j £ J' is from a specific route type r(j) £ I. In 
particular, if we let r(j) be the route type of the customer at the head of queue j £ J' and we let the vector 
n' give the number of customers of each route type at the heads of these queues then we can see that 

*r(fc(l) = r(j),j £ J'}\n) = E Wn) U U ^ 



fJ-jr(j) 



n 



y — 

^ Bin. 
c'ex(n-n') v ' jeji-.jei 



II II ^ 



B(n-ri) 
B{n) 



In the second inequality above, we factor out the multiplicative terms corresponding to the heads of each 
queue in J' . We then notice the remaining states that must be summed over are all the states where there 
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are n — ri customers on each route. The resulting sum then gives the normalizing constant when there are 
n — n' customers on each route. 

Next, by Proposition [2] we have that 



=r(j),j€j'}\n) = 



B(n-n') 

x — V,, \ > 

B(n) 



n 

jej' 



(4.6) 



The events {ij(l) = r(j),j £ J'} are disjoint for different choices of r' = (r(j) : j £ J 1 ). For a queue to be 
nonempty, there must be some customer at its head. Thus, 



and consequently using (|4.6p . we have 

7r({mj>0,iG l 7}|n)= ]T 7r(ft(l) = r(j), j e j'}\n) 
r'ex- 1 ' 

/ B(n-n') 



= e 



r'6I J ' J'EJ 

— e n 



B{n) 



n e 

jer Vex 



A*(n) 

I'jr 



: J] AJ(n) 



as required. 

Now we are in the position to prove the convergence of term (c) in expression (|4.3[) 
Proposition 3. 



B(cn + k c 
B(cn + k c 



n a-^w) 



□ 



Proof. In the following expression, we use Lemma 0] we apply the Inclusion- Exclusion Principle (I4.8|l ; we 
apply Lemmal5l (|4.9[) ; and then we notice the resulting summation is I~[j£,7° (l — A j ( n )) expanded. Formally, 
we obtain 



_B(en + fee) 
B(cn + kc) 



=?r({m.j = 0, Vj E J"°}|cn + fe c ) 
=l-7r( IJlmj > 0}|crc + fc c 



(4.7) 



ji<---<j'fe 



e e n a»= n (i-^-w)- 



k =°31, — ,3k£J°: 3=31, — ,3k 

31< — <3k 

In expression 114.8I) . the fc = summand is understood to equal 1. 



(4.8) 



(4.9) 



□ 



We have now discovered the limiting behavior of terms (a), (b), and (c), and we can prove Theorem [T] 

Proof of Theorem^ The result holds by applying Proposition [2] and equality (|4.5p . which we derived from 
Proposition [21 to our derived equality (|4.3p : 



o/ oi i , \ B(cn + k c ) B(cn + k c -n°) 

tt (m \cn + k c ) =—± r x — — '- x 

v 1 ' B(cn + k c ) B{cn + k c ) 



n 



II m° 

i--3&i Hji 



n (i-A*(n))x n n A ?w m?i x n 



< : i e * 



n 



Ai(w) 



= n a-^w) >< n 

=7r A »( n) (m ). 



: j E i 



n 



Aj(w) 



This proves that our network's non-bottleneck queues become independent. 



□ 
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4.2 Proof of fluid limit. 

Let = {i 6 1 : j ' € i} and ^ \T(j)\. Assuming that set is ordered, we also denote by 
with I — 1, . . . the function that enumerates its elements. We define for each j £ J , an independent 

Poisson marked point process Nj with intensity /x, dt <g) du on R x [0, 1], where fij = max{i € : /ijj} 
and the function Xj( m 3i 

u) on N I(J) x (0, 1) in the following way: 





ij (0 if Eh=o m jh Mjh < rrij fi 3 x u < Y^ h =o m ^ Pft 
I if J2h=o m Jh Hjh < rrij \ij x u < mj fij. 



Therefore, the network process can be written as 

dMn(t) = l{ Xj '(M&-),U N ., (t) ) = i}dN r (t) - l{ X] {M 3 {t-),U N]{t) ) = i}dNi(t) 

where j' denotes the queue just before j on route i. In the following we will use also the notations j and 
f to denote the first queue on route i before and including the queues j and j' respectively that are non 
empty. In integral form, we have 

Mjiit) = Mji(O) + Cj'S) - Cji(t) + A jH (t) - Aji(t) 

where £ji(t) is the martingale 

(*) = / / HXj (Mj («-),«)=»} [dNj (s, u) - ftj du ds] , 
Jo Jo 

and the process Aij(t) is given by 

Aj<(t) = / / ■HXj(Mj(s-),u) = ijduds. 
JO Jo 

From the equation above, noticing that J Q l-fjfo- (M,- (s, w) = = {fiji Mji(s))/(fij Mj(s)) and with the 

assumption that 0/0 = 0, we get that 

— Aji(t) = f— ^ l{Xi(Mi(s-),u) = i}dud« = f (a ~ } tfa. 
io W. Jo io 

Summing over i : j £ i we have that for f' < t" 

^2±(A ji (t")-A ji (t')) = f ^M^=lds=[ t *£l{Mj(s-)^0}ds<t"-t', (4.10) 

which gives the Lipschitz condition for the process Aji(t). 

We define the scaled process M(nc, t) = c _1 M(nc, ct), such that, having M,j(0) = c%i, it has the 
j'i-component given by 



Mji(nc, t) = riji + 



Cj'i{cb) ~ Cji(ct) Ljn{cb) - Lji(ct) 



c c 

Using the same steps as before, we can rewrite the process c~ 1 Aji(ct) in the following way 
Aji ^ - ' I I fJ , j l{ Xj {M j (s),u)=i}duds 

•* M 3t {cs) 



'o Jo 

ft /■! 



/ / fJ.jl{Xj(Mj(cs),u)=i}duds= —f- — - ^ ds 
Jo Jo Jo Mj(cs) 



Proposition 4. Given n 6 N Jx/ , the processes {(M(nc, t),t > 0)} c >o, with M(nc, 0) = nc, are tight. 

Proof. By the triangular inequalities for metrics, to prove tightness of the multidimensional process {(M(nc, t),t > 
0)} c >o, it is enough to show that any of its coordinate processes are tight. By Theorem C.9 in Robert [28] . 
this can be done by showing that given ij, for any fixed T, r\ > 0, there exists 8 > such that 

Pr{^. i( „ e ,.)W >V}<* (4-H) 
where, for a given function /, the modulus of continuity on [0, T] is defined as 

u f {5) = sup{ S ,t < T,\t — s\ < 6 : \f(t) - f(s)\}. 
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Since Mji(nc, t) = n+c" 1 A£ 3 -i(ct)+c _1 AAji{ct)-(Cji{ct)) with A£ji(i) = Cjn^-Cj^i) and AA^t) = 
Aj'i(t) — Aji(t), it is enough to prove that relation 1)4.111) is valid separately for the processes c -1 A£ji(ct) 
and c _1 AAji(ct). We have 



Pr 
Pr 



| |AA^ ct )_AA ij ( CS )| >7? | (412) 

r I /•* fM jH (cu) M ]r (cu) \ I i 

- Pr { .,< r T.i<* / l^r ^ ~ H du > "} 

< Pr{ sup 2]i - s|/x > 77} = (4.13) 

with 5 < rj/(2/i) and where /i = max{i, j : /iji}. 
For the martingale c~ ACji(ct), we have that 

PH sup — — '- — — '- > r\ \ = Pt< sup — — - > ^ } < —--E \{ACji(ct)) \ 

a,t<T;\t-s\<S I c C I J I t < T C 2 J C<V 

where the last inequality follows by applying the Doob's inequality. Having that E [(A£ji(cT)) 2 ] < 2/xcT 
we have that for c > 8/iT/(e»7 2 ) the probability is bounded above by e, as required. 

□ 

The tightness property ensures the relative compactness, therefore from every sequence {(M(nc, t),t > 
0)} c >o, with M(nc, 0) = nc and n fixed, it is possible to extract a convergent subsequence. The following 
proposition ensures that any limit process will be given by a fluid solution. 

Proposition 5. Assume that a sequence of processes {(M(nc, t),t > 0)} c >o converges to a limit process 
m(t) as c — !> oo. Then, m(t) is almost surely continuous and it is a fluid solution, i.e., it satisfies the 
following conditions 

mji (t) = A* jH (t) - A*i(t), (4.14a) 
— (Aj'iW - A?((a)) <t-s, (4.14b) 

\ji(t) is increasing, (4.14c) 
dA*At) in™ 

ifmj > then — 2il2 = _ (4.14d) 
at nij 

y^mjift) = Uj. (4.14e) 

Proof. Using the Skorohod's Representation theorem, see Robert [281 Theorem C.8], we can assume that all 
the elements of the sequence are random processes defined on the same probability space with probability P 
and the convergence is P-a.s. 
Since we have that for any c 

M Jt (nc, t) = n H + + _ M^, 

c c c 

passing to the limit and using bounded convergence for the integral we get 

fhji{t) = riji + A*,i(t) - A*i(t) 

where 

A H (t) = lim / — — — r njidu, 
JlV ' c^oo J Mj(cu) J 



which exists by (|4.12|l . In particular, if nij(t) > for some t, it will be positive in a neighborhood B(t) of t 
by continuity. It follows that for t' < t" and t' , t" € B(t), 

a* u"\ a * t ,'\ v r Mji(cu) f* .. Mji(cu) f* mjiiu) 

A H {t ) - A H {t ) = km / m du = hm Hi du = ^ Hi du, 



c— >oo 



where in the second equality we have used the bounded convergence theorem, which implies 

dtAUt) = Hi mji ¥} asm J -(t)>0. 
mj(t) 

The additional conditions satisfied by A|j(t) easily follow by the property of the approximation processes 
{c _1 Aji(ct),c > 0}, in particular the Lipschitz condition follows by (|4. lOf) . □ 
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4.3 Proof of convergence of bottleneck queues. 

We now focus on proving Theorem [3] Our proof is quite involved and requires a number of lemmas. 

For a fluid solution m(t), A(t), let A^(t) be the derivative of Aji(i), when it exists. Let also Iji be the 
queue before queue j on route i and Xji be the next queue after queue j on route i. 

The following proposition and lemma helps us differentiate our Lyapunov function. 

Proposition 6. For any time interval [to,t] with t > to > 0, there exists a constant D > such that for 
any ti, t 2 6 [to, t] 

\P(t 2 ) -/0(ti) | < £>|*2 -*,.| . (4.15) 
Lemma 6. For a function x : R+ — > R+, if t > is such that the derivative of x exists at t, the derivative 
of x log x exists at t and x(t) = then 

dx dxlogx 

— = ana = 0. 

dt dt 

Proposition [S] is based on Proposition 4.2 of Bramson [5]. Both Proposition |S] and Lemma HJ] are proven 
in Appendix[B] Recalling that Xji is the next queue on route i after j and that Iji is the queue before queue 
j on route i, we can now prove the following proposition. 

Proposition 7. For almost every t, 

S-EEMlft (4.16) 

Proof. Our processes in Proposition [5] are absolutely continuous and thus almost everywhere differentiable. 
So for almost every t, we can differentiate the terms Aji(t), m^-(t), rrij (t) , my (t) log rrijiit), mj(t)logrrij(t) 
and P(m(t)). We now differentiate the function 

P(m(t)) = 2~I E ( m ^W logmjj(t) - m 3i logitjij - ^ rrij(t) logmj(t). 

We obtain 

d/3(m(t)) y- y- ( dmqlogmji dm di \ y- rfmj log m, 

jeji:j£i jej 

y- y^ /drnjilogniji dmji \ y- dnijlogmj , . 

-EE ( — s dr log ^)- E — ( 4 - 17b ) 

m ji (t)>0 m j (t)>0 

x — - x — 1 ( dmn , dm™ dm,i , \ \ — 1 /" dm, , dm, \ < . 

= E E hf log ^ + ^f -^r log ^)- E (^r log ^ + ^r) ( 4 - 17c ) 

m ji (t)>0 m 3 -(t)>0 

jEJ i:jei JV y 

m ji (t)>0 

=EEK*( { ) - A ^w) lo s A ^w ( 4 - 17e ) 

= EE A ^(*)( lo s A ^^(*)- lo s A ^(*)) ( 4 - 17f ) 

=-ee a ;.w'^fi)- (4 - i7g) 

In the above sequence of equalities, equality (14. 17b [1 holds by Lemma[6] Equality (|4.17d|) holds by observing 
that X^i jgi m js = m j an d canceling terms. For equality l|4.17ep . we know by our fluid model assumption 
(|2.9a|l that m'ji(t) = Aj^ft) — A^(£). In addition, we note that if rriji > then, using (|2.9d|) . log = 
logAL(f) and if m,ji(t) = then = m'^it) = Aj.^t) — Ajj(t) and Ajj(i) > 0. Thus, we may reintroduce 
the rriji = terms in our summation. In equality (|4.17ep . for each route, we re- interpolate the first term in 
our summation. □ 

The next lemma, found in Cover and Thomas |5], will be of key importance in bounding our Lyapunov 
function. 

Lemma 7 (Pinsker's Inequality). For the relative entropy between two discrete probability distributions 
V = (Pj)j an d 1 = with the same support: 

D(p\\q)=J2PitoS*T> 

the following inequality holds 
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Applying Pinsker's inequality to Proposition [7] gives 
Lemma 8. For almost every t, 

where \ J\ is the size of set J and we recall that /J. m ax = maxj/iji : i 6 X,j 6 i}. 

Proof. Let f be a time for which Proposition [7] holds and let Af(t) = 53 g, Aji(t), for i £ I. For each i, let 
= Aj-j (i)/Af (t) and qj — A' Xjii (t)/Af (t). By Lemma [TT1 p and g both have the same support. For each 
i, we applying Pinsker's Lemma 

g H * If - § K bs I Kg - •')" * g - - wW g ( a; ' w - A - (,) 

Multiplying the left and right of this inequality by — Af(t), summing over i £ I gives 

Recall that from Lemma that AL(t) < /i maa: thus Af(t) < \J\fJ, m ax- Applying this bound to Af(t) the 
above equation gives the required result (14.1811 . □ 

We define m* to be a solution to the optimization problem 

minimize /3(m) subject to ^3 my = ru, i £l over 77%,-j > 0, i 6 1, j 6 i. (4-19) 

As we discussed we expect the path of the m(t) to converge to the optimal value of the optimization. To 
conduct further analysis, we characterize the dual of this problem. 

Lemma 9. The dual of the optimization (|4.19p is 

maximize ^^n;logA; subject to ^3 < 1 over A, > 0, i £ 1. 

Proof. Taking Lagrange multipliers A £ R 7 , its Lagrangian is, 

L(m, A) = ^2 ^2 m ji l°g mj ^ : " + 53 I n i - 53 m ji ) 

= 53 E m ^ lo s^xr + E A ^ 

= 53 m i- D (P J 'll9 J )+ 53 m J lo S ( 53 ^Vji) +5Z Ain; - 

In the final, inequality above we let p 3 = (rnji/rrij : i B j) and g J = (e jU^ /2r e * r ^Jr : * ^ ■?)■ R- ecamn g 
our Remark[2]on relative entropies, this Lagrangian is minimized by taking p J = q 3 for each j £ J and then 
by minimizing over rrij. In particular, we get 

minL (m ,A) = f E ->» n ' Al * E — £ * h J > 
mes^ I -co otherwise, 



thus we find dual, 

maximize 53 n * Ai subject to 53 ~ — — 1 over A £ R 7 . 
Substituting Aj = e A » gives the required result 



maximize 53 n il°gAi subject to 53 — ^> ^ ^ over ^ ^ 

i:n^ >0 3 * 

□ 

Lemma 10. //, /or some e > 0, fi{m(t)) > /3(m*) + e then there exists 5 > 0, i £ I and j £ z suc/i t/iat 

lA^^-A^i^)! ><5. 
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Proof. We develop a proof by contradiction. If this result was not true, as the set of queue sizes is compact, 
we could construct a sequence of times t k , k = 1,2,3, ... such that f3(m(t )) > /3(m*) + e and m(t ) — > rh, 
as k — > oo and 

^ — ' ^ — ' I I fc— >oo 

For each queue j £T with m,- > 0, let j 8 + be the next non-empty queue on route i i.e. > 0. We can 

say 



A'(t fc ) and A'.+ .(* fe ) 



m .+ u .+ . 



We letting J + be the set of queues on route i between j and jf that includes j but does not include jf 
Applying a triangle inequality across these queues, we can say that 



< lim 

k — >oo 



™ m .+ .u .+ . 



A;+ 8 (t fc ) 



+ 53 |Aii(t fc )-A; Hl (t* 



Applying this triangle inequality once more, for any queue I on route i that is between j and j'J" , we see that 



lim 

k — >oo 



In other words, for each route i for all queues j G i, A'j i {t k ) converges to some value A; > where if rhj > 
we have that 



for some constant Aj > 0. Observe that, by (|2.9b|) . for each queue j G J 

11 ji k— oo M 7 i 

also 



= X] "^iilogAi = 5^n*logAi. (4.20) 

Thus, the vector A = (Aj : i G I) is feasible for the dual problem and the vector rh is feasible for the primal 
problem. We know by Weak Duality (for a minimization) that the any primal feasible solution is bigger 
than that of the dual. Thus, we know by (|4.20fl that the primal equals dual solution and so rh must be 
optimal for the primal problem i.e. f3(rh) = /3(m*). This must be a contradiction because by assumption 
j3(m(t k )) > P{rn*) + e and thus by continuity of /?, /3(rh) > /3(m*) + e. □ 

We are now in a position to prove Theorem 

Proof of Theorem^ We found f3(m(t)) was absolutely continuous in t. In Lemma[8] we found the derivative 
of P{m{t)) was almost everywhere negative and thus /3(m(t)) must be a decreasing function. 

Suppose for s G [0, t], /3(m(s)) > P(m*) + e for some e > then by Lemma [TO] there exists an i G I and 

\K' ji {s)-h.' ji {s)\>5„ 

for some S € > 0. Thus applying this to our bound in Lemma for intervals of time [0, t] such that /3(m(s)) > 
P{m*) + e, we have that 

P(m(t))<p(m(0))-t 5 ■ 

As p(m(t)) is bounded below by /3(m*), the above inequality cannot be sustained for all times t. In other 
words, eventually /3(m(t)) < /3(m*) + e. Thus /3(m(t)) \ /3(m*). This proves the first assertion in Theorem 

El 



Now, it remains to show that m(t) approaches M, the set of solutions to H4.19[) . Take some ex > 0. Let 



m — (rriji : i G I, j G i) be any vector with 



n-i for i 6 1 and such that 



min \m — m > ei. 



Such an m belongs to a compact set and thus, as ft is continuous, it must be that /3(m) > fi(m*) + e for 
some e > 0. Or stated differently if /3(m) < /3(m*) + e then it must be that 



mm \m — m \ < ex- 



As we have just shown f3(m(t)) < /3(m*) + e holds eventually for all fluid paths. Thus, 

lim min \m(t) — m*\ = 0. 



□ 
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A Proof of Lemmas [T] and [21 



Proof of Lemma\T\ Taking j = j{, we have 
Aj(n) = u,j — — 7r(m|n) 

mGS(n): J 
m,->0 



meS(n): 3 K ' IGJ \ V 7 r:i6r V ^ 



m~ >0 



RfVi II ( ( m ' ■ r a 7 ) II ( ,„ 



In the third inequality, we canceled terms and substituted m' lr — mi r — 1 if (l,r) = and m' lr — mi r 

otherwise. □ 

Proof of Lemma\^ We consider both optimal solutions A*(n) and A*(n). Let 

G„(A) = ^n l logA l 

iex 

Since A*(n) is the solution of an optimization with a larger feasible set G n (A*(n)) > G n (A*(n). 
Take v — A*(n) — A*(n). Note A*(n) + Sv belongs to feasible set 



{a>0:£^<1 



for all S suitably small. If this was not so then their would have been some constraint/queue which we did 
not correctly include in the set of bottleneck links J. Taking the partial derivative of G„ from A* (n) in the 
direction of v, we can then say that 

v 0G„(A») 

iex 

This holds because A*(n) is optimal. Now, also, by the concavity of G n (-) 

9G n (A*(n)) 



G n (A*(n))-G„(A*(n)) < 



dAi ■ 

So G„(A*(n)) < G„(A*(n)). So G„(A*(n)) = G„(A*(n)). By strict concavity of G„(-), A*(n) is the unique 
feasible solution it's optimization problem. Thus as A*(n) is also feasible it must be that A*(n) = A*(n). □ 

B Lipschitz Continuity of (3(m(t)). 

Before proceeding to prove the Lipschitz continiuty of /3(m(t)), We shall quickly give a proof of LemmaJS] 

Proof of Lemma\B[ We use the fact that we know that the derivative exists. Firstly, it is clear % = 
because, 

dx , x(t + h)-0 , dx , xft + A) — 

— = lim v — ■ ; > 0, and — = lim v , ; < 0. 

dt h\,o h dt h/'o h 

S° — 0- Noting that a;log(a;) is negative for all x < 1. By essentially the same argument 

dx log x x(t + h) logx(t + h) — dxlogx a;(i + /i) log x(t + h) — 
: = lim : v U, and : = lim : ^ (J. 

dt hNO h dt hyo h 

Thus =0. □ 

at 

We now demonstrate that the function j3{m(t)) is Lipschitz continuous on any compact time interval. 
Here m(t) is any solution to the fluid equations (|2.9[) and /3(m) is defined by ()3.5|) . The following arguments 
are adapted from Lemma 4.2 and Proposition 4.2 of Bramson [5]. All queues may empty in our network, so 
we have to apply some degree of care in proving the Lipschitz continuity on compact time interval. 

Lemma 11. For almost every t, < Aj 4 (f) < fXmax, where \x m a X = max{/i^i : i £ I, j £ i}. 

Proof. Suppose that m(t) and A(t) are differentiable at t. We may assume A'jj(t) > for some queue j on 
route i. Such a queue must exist because there is always some queue with m,ji(t) > as rii > and thus by 
()2.9dl) A'jiit) > 0. Now consider Xji the next queue on route i, if m Xji i(t) > then by (|2.9d[l A' Xjii > 0, and 
if rn Xji i(t) — then m' Xjii (t) — 0, thus by (|2.9a,f) A^.^t) = Aj,(t) > 0. Continuing inductively we see that 
Aj-j(t) > for all queues. 

From this argument we now see that the value of A^(t) on any route i is achieved by a queue j* with 
m 3 *i > 0. Thus applying (|2.9d|) . A^(t) < 
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Proposition 8. For almost every t > to > 0, there exists a constant T > such that if t — to < kT for 
k G N then 

xahiji Ki(t ) 



mm i 

ji 



Mjiit) > 



(1 + maxji fj,ji 



Here c m H ) is a strictly positive constant which depends on m(to), the fluid model state at time to- 
la order to prove this proposition we require the following lemma 

Lemma 12. For almost every time to and t with t > to, if a queue j on route i, has arrival process from 
the queue before j, Iji such that for almost every s £ [to,t] 

then, almost everywhere, the output of route i work from queue j satisfies 

A' 3l (t)>c' (B.l) 

where 



c — mm 



A^i(io), 



1 + maxji 



Proof. We note that Ay (t) is a Lipschitz function and thus is almost every where differentiable. We assume 
that t and t' are differentiable points where (|B . 1 f) is violated. Observe that if rriji(t) = then, by ()2.9al) . 
= m'jiit) = Aji(t) - Aj. 4i (t). Thus A^(t) = Aj ..^(t) > c> c . So it must be that mji(t) > 0. Consequently 
by (|2.9d[l . A^(s) = fj,jimji(s)/mj(s) must be continuous on an interval around t and there must be an open 
interval arround t for which A' j(s) < c . 

For a AL(s) to get small we need the total number of departures to be comparible relative to the arrivals. 
So, we will next argue the contradiction that Aj-j(i) cannot enter an interval of time for which A'j i (s) < c' 
without the average departure rate (A^(t) — A^(s))/(t — s) being bigger that c'. 

We let t be the last time before t and after to for which Ajj(t) > c . Note Ajj(io) > c', so t is well defined. 
We use the shorthand Aji(i,t) = Aji(t) — Aji(i) and Aj(t, t) = rtij(t) — rrij(i). As mij(t) > 0, by (|2.9d|) . we 
have 

> ^ \> m m Jl (t) + A l i(t,t) -Aji{t,t) 
c > A,v(i) = M« — ^-^N 1 = Mil ^ = ■ 

Rearranging the above expression implies 

Aji(t,t) > A h%i (t,t) - c'Aj^t) + MimiS) ~ cm 3 (i) > A hii (t) - c'Aj(i,i). 



The last inequality holds because, by ()2.9d[l . A^(f) > c implies [ijirtiji(t) — c'mj(t) > 0. Let's now look at 
the mean value of the terms in the above inequality: 

Aji(i,t) ^ At i(t,t) ,Aj(i,t) , 

. > — = c — . > c — c max ixn > c . 

t-t t-t t-t a 

In the second inequality above, we use the assumption that A|.^(s) > c, s £ [to, to + T] and the fact the 
average change in rrij(t), Aj(t,t)/(t — t), is at most by the maximum service rate maxji/iji. The final 
inequality holds by our choice of c . This then contradicts our assumption that A'j i (s) < c on the interval 
(t,t}. □ 

The following lemma is a consequence achieved by iteratively applying the last result. We have to apply 
some care because, in comparison to Bramson's open network analysis [5], every queue can empty or have 
low arrival and departure rate. 

Lemma 13. For almost every to, there exists an interval of fixed length [to, to + T] such that for almost 
every t € [to,to + T] and for all i £ X and j 6 J 

A'j.it) > c(to) > 0. 
Here c(to) is a function of the network's state at time to- 
Proof. On route i there is always one queue with greater than or equal to the average amount of work. With 
out loss of generality, i.e. relabelling queues if necessary, we assume that this is the first queue on route i. 
So, we have that for any route i 

min r n r 
m^So) > — - — . 

The biggest rate that this queue could decrease by is max.,; fiji. So 

. . min r n r „ . 
m s i t (t) > 2/ for t e [to, t + T], 



where we define T : 



27 maXji iiji 



a u \ rnjifto) _ min^i /iji min r n r 

Aji(t ) = ^ > = Cl . 

m,j(to) 2l}_, n T 
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We can now repeatedly apply Lemma (|12|l . Starting from ci above for k = 1, k% — 1 we define 

c k+1 = min <^ A'jiito), — — > ■ 

Each time we iterate we reduce by at least 1 + maxji/iji. A simple lower bound, which is sufficient for 
our purposes is that 

Cfe > miiiji A'jjjto) = _ 
~ (1 + maxji Hji) K 

where K — max, ki is the longest route within the queueing network. Thus given this, we have from Lemma 
(fT2l) that 

A'jiit) > c(t ) 

for almost every t € [to , to + T\ . □ 

Here in a similar manner to Proposition 4.2 of Bramson 5] we show that /3(t) is Lipschitz on any compact 
time interval. 

Proof of Proposition^ Note that without loss of generality, we may assume interval [to,t] is of length less 
than or equal to T = 2 imTx ' wnere ^ was dervived in the last lemma, Lemma If 31 If t — to > T then 
we can split the interval [to,t] in to overlapping sub- interval of size T and then use the largest Lipschitz 
constant found in each sub-interval as a Lipschitz constant for [to-t]. 
Note that 

P(m(t)) = EE m^log ^g^ J2 E ^(i)logA^(i)- 
It is enough to prove Lipschitz continuity of each term summed above: 

\ma{ta) log(A^(i 2 )) - rmjifx) log(A^(ti))| < Di |i 2 - *i| . 

By Lemma IT3l for s £ [to,t], log(A^(s)) is bounded below by c(to) and also above by Cj, the capacity of 
queue j. So 

|log(A^( S ))| < .Do : =max{[log(Q)|,[log(c(t ))|. 

If rriij(t 2 ) = rriij(ti) — 0, the relation is trivial, so we assume that rriij{t 2 ) > 0. If TOy(ti) = we have 
that 

\m^{t 2 ) log(A^(i 2 ))| < Do\m lJ {t 2 )\ = Do\m lJ {t 2 ) - m lj (ti)\ < D 2 D \t 2 - ti\ 

where the constant Do is as above and the constant D 2 is the Lipschitz constant of rriij (t). 
Now assume my(t 2 ) and rriij (ti) both positive, and without loss of generality that 

A^(ti) < A'jtfa) . 

It follows that 

\rriij(t2) log A'ji{ti) - m i3 (ti)logA^(ti)j < \mij(t2) - rmj(ti)\ |log Aji(t 2 )| 

+ m iJ (t 1 ) |logAj-i(t 2 ) - logAji(ti)j 

Again the first term in this upperbound is less than or equal to D 2 Do \t 2 — The second therm, since 
the logarithm function is concave and has its derivative maximized at the left point, can be bounded in the 
following way 



■ rriij (ti 



< 


rriij (ii) 


AJ 4 (ti) 




, . i 
mj{ti)- 


< 


rriij (t 2 ) 


m,j(t 2 ) 


< 


mj(t 2 ) 


< D 3 \t 2 - 



mj(t 2 ) 

\m,j{t 2 ) - m,j(t±)\ + \mij(t 2 ) — mij(ti)\ 



that follows from the fact that mij(t 2 ) < rrij(t 2 ) and the fact that both rriij(t) and rrij{t) are Lipschitz 
continuous. The result follows by choosing D\ > max{Z>3, D 2 Do}. □ 
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